Patient and public involvement in the design, administration and evaluation of patient feedback tools, an example in psychiatry: a systematic review and critical interpretative synthesis.

BACKGROUND: Patient feedback is considered integral to healthcare design, delivery and reform. However, while there is a strong policy commitment to evidencing patient and public involvement (PPI) in the design of patient feedback tools, it remains unclear whether this happens in practice. METHODS: A systematic review using thematic analysis and critical interpretative synthesis of peer-reviewed and grey literature published between 2007 and 2017 exploring the presence of PPI in the design, administration and evaluation of patient feedback tools for practising psychiatrists. The research process was carried out in collaboration with a volunteer mental health patient research partner. RESULTS: Fourteen articles (10 peer-reviewed, four grey literature) discussing the development of nine patient feedback tools were included. Six of the nine tools reviewed were designed from a professional perspective only. Tool content and its categorization primarily remained at the professional's discretion. Patient participation rates, presence of missing data and psychometric validation were used to determine validity and patient acceptability. In most instances, patients remained passive recipients with limited opportunity to actively influence change at any stage. No article reviewed reported PPI in all aspects of tool design, administration or evaluation. CONCLUSIONS: The majority of patient feedback tools are designed, administered and evaluated from the professional perspective only. Existing tools appear to assume that: professional and patient agendas are synonymous; psychometric validation is indicative of patient acceptability; and psychiatric patients do not have the capacity or desire to be involved. Future patient feedback tools should be co-produced from the outset to ensure they are valued by all those involved. A reconsideration of the purpose of patient feedback, and what constitutes valid patient feedback, is also required.

The impact of patient feedback on the medical performance of qualified doctors: a systematic review.

BACKGROUND: Patient feedback is considered integral to quality improvement and professional development. However, while popular across the educational continuum, evidence to support its efficacy in facilitating positive behaviour change in a postgraduate setting remains unclear. This review therefore aims to explore the evidence that supports, or refutes, the impact of patient feedback on the medical performance of qualified doctors. METHODS: Electronic databases PubMed, EMBASE, Medline and PsycINFO were systematically searched for studies assessing the impact of patient feedback on medical performance published in the English language between 2006-2016. Impact was defined as a measured change in behaviour using Barr's (2000) adaptation of Kirkpatrick's four level evaluation model. Papers were quality appraised, thematically analysed and synthesised using a narrative approach. RESULTS: From 1,269 initial studies, 20 articles were included (qualitative (n=8); observational (n=6); systematic review (n=3); mixed methodology (n=1); randomised control trial (n=1); and longitudinal (n=1) design). One article identified change at an organisational level (Kirkpatrick level 4); six reported a measured change in behaviour (Kirkpatrick level 3b); 12 identified self-reported change or intention to change (Kirkpatrick level 3a), and one identified knowledge or skill acquisition (Kirkpatrick level 2). No study identified a change at the highest level, an improvement in the health and wellbeing of patients. The main factors found to influence the impact of patient feedback were: specificity; perceived credibility; congruence with physician self-perceptions and performance expectations; presence of facilitation and reflection; and inclusion of narrative comments. The quality of feedback facilitation and local professional cultures also appeared integral to positive behaviour change. CONCLUSION: Patient feedback can have an impact on medical performance. However, actionable change is influenced by several contextual factors and cannot simply be guaranteed. Patient feedback is likely to be more influential if it is specific, collected through credible methods and contains narrative information. Data obtained should be fed back in a way that facilitates reflective discussion and encourages the formulation of actionable behaviour change. A supportive cultural understanding of patient feedback and its intended purpose is also essential for its effective use.

Responding effectively to adult mental health patient feedback in an online environment: A coproduced framework.

BACKGROUND: Responding to online patient feedback is considered integral to patient safety and quality improvement. However, guidance on how to respond effectively is limited, with limited attention paid to patient perceptions and reactions. OBJECTIVES: To identify factors considered potentially helpful in enhancing response quality; coproduce a best-practice response framework; and quality-appraise existing responses. DESIGN: A four-stage mixed methodology: (i) systematic search of stories published on Care Opinion about adult mental health services in the South West of England; (ii) collaborative thematic analysis of responses to identify factors potentially helpful in enhancing response quality; (iii) validation of identified factors by a patient-carer group (n = 12) leading to the coproduction of a best-practice response framework; and (iv) quality appraisal of existing responses. RESULTS: A total of 245 stories were identified, with 183 (74.7%) receiving a response. Twenty-four (9.8%) had been heard but not yet responded to. 1.6% (n = 4/245) may lead to a change. Nineteen factors were considered influential in response quality. These centred around seven subject areas: (i) introductions; (ii) explanations; (iii) speed of response; (iv) thanks and apologies; (v) response content; (vi) signposting; and (vii) response sign-off that were developed into a conceptual framework (the Plymouth, Listen, Learn and Respond framework). Quality appraisal of existing responses highlighted areas for further improvement demonstrating the framework's utility. CONCLUSION: This study advances existing understanding by providing previously unavailable guidance. It has clear practical and theoretical implications for those looking to improve health-care services, patient safety and quality of care. Further validation of the conceptual framework is encouraged.

Optimizing patient and public involvement (PPI): Identifying its "essential" and "desirable" principles using a systematic review and modified Delphi methodology.

BACKGROUND: There is international interest in the active involvement of patients and the public. However, consensus on how best to optimize its application is currently unavailable. OBJECTIVE: To identify and assess the underlying principles of patient and public involvement (PPI) in health and social care services, research, education and regulation across medicine, dentistry and nursing. DESIGN: A four-phase methodology: (i) an extensive systematic review of published and grey literature; (ii) inductive thematic analysis of review findings; (iii) development of best practice principles; and (iv) consensus testing of identified principles using a modified Delphi methodology. SETTING AND PARTICIPANTS: Twelve systematic reviews and 88 grey literature publications were reviewed leading to the unique identification of 13 principles later assessed by 18 PPI experts. RESULTS: Essential consensus (>75% agreement) was obtained for nine principles reviewed. Working in equal partnership and sharing information achieved the highest consensus rates: 16/17 essential 94.1%; 1/17 desirable 5.8%. The four remaining principles that failed to reach essential consensus were categorized as desirable by expert respondents. No principles were considered irrelevant. No alternatives were suggested. DISCUSSION: Expert respondents suggest essential principles must be achieved to optimize PPI best practice. To advance PPI practice, desirable principles should also be aspired to wherever possible. CONCLUSIONS: This study's innovative approach advances existing knowledge by providing previously unavailable consensus about PPI best practice. Research findings hold important theoretical and practical implications for educators, regulators, researchers and practitioners looking to effectively work together.

The McDonaldization of appraisal? Doctors' views of the early impacts of medical revalidation in the United Kingdom.

INTRODUCTION: Medical regulation is rapidly changing with claims that systems such as revalidation/relicensing will reassure the public. Yet the impact of such initiatives is unknown. METHODS: Using the principles of efficiency, calculability, predictability and control through technology, identified by Ritzer, and exampled by the McDonalds business model, we analyzed interviews with doctors between May 2012-Dec 2013 which focused on doctor experiences of appraisal and revalidation in SW England. RESULTS: The research found significant changes in appraisals since the launch of revalidation in December 2012. Appraisal has been standardized with a list of supporting information that must be collected by doctors. The success of implementation is measured in the numbers of appraisals completed but less is known about the quality of the appraisal itself. Such efficiencies have been supported by IT systems that themselves might be at risk of driving the process. DISCUSSION: There are potential advantages to McDonaldization including appraisals available to all, not just for doctors working in the NHS, and a potentially more appetizing recipe for their completion. As yet a state of McAppraisal has not been reached; with a complete transfer of trust in the doctor to trust in the appraisal process within revalidation. However policymakers will need to continue to ensure that regulatory initiatives, such as revalidation, are not just a process for their own sake.

The impact of large scale licensing examinations in highly developed countries: a systematic review.

BACKGROUND: To investigate the existing evidence base for the validity of large-scale licensing examinations including their impact. METHODS: Systematic review against a validity framework exploring: Embase (Ovid Medline); Medline (EBSCO); PubMed; Wiley Online; ScienceDirect; and PsychINFO from 2005 to April 2015. All papers were included when they discussed national or large regional (State level) examinations for clinical professionals, linked to examinations in early careers or near the point of graduation, and where success was required to subsequently be able to practice. Using a standardized data extraction form, two independent reviewers extracted study characteristics, with the rest of the team resolving any disagreement. A validity framework was used as developed by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education to evaluate each paper's evidence to support or refute the validity of national licensing examinations. RESULTS: 24 published articles provided evidence of validity across the five domains of the validity framework. Most papers (n = 22) provided evidence of national licensing examinations relationships to other variables and their consequential validity. Overall there was evidence that those who do well on earlier or on subsequent examinations also do well on national testing. There is a correlation between NLE performance and some patient outcomes and rates of complaints, but no causal evidence has been established. CONCLUSIONS: The debate around licensure examinations is strong on opinion but weak on validity evidence. This is especially true of the wider claims that licensure examinations improve patient safety and practitioner competence.

Towards a pedagogy for patient and public involvement in medical education.

CONTEXT: This paper presents a critique of current knowledge on the engagement of patients and the public, referred to here as patient and public involvement (PPI), and calls for the development of robust and theoretically informed strategies across the continuum of medical education. METHODS: The study draws on a range of relevant literatures and presents PPI as a response process in relation to patient-centred learning agendas. Through reference to original research it discusses three key priorities for medical educators developing early PPI pedagogies, including: (i) the integration of evidence on PPI relevant to medical education, via a unifying corpus of literature; (ii) conceptual clarity through shared definitions of PPI in medical education, and (iii) an academically rigorous approach to managing complexity in the evaluation of PPI initiatives. RESULTS: As a response to these challenges, the authors demonstrate how activity modelling may be used as an analytical heuristic to provide an understanding of a number of PPI systems that may interact within complex and dynamic educational contexts. CONCLUSION: The authors highlight the need for a range of patient voices to be evident within such work, from its generation through to dissemination, in order that patients and the public are partners and not merely objects of this endeavour. To this end, this paper has been discussed with and reviewed by our own patient and public research partners throughout the writing process.

"No one has yet properly articulated what we are trying to achieve": a discourse analysis of interviews with revalidation policy leaders in the United Kingdom.

PURPOSE: To analyze prevailing definitions of revalidation (i.e., a recently instituted system of ongoing review for all physicians in the United Kingdom), the circumstances of their origin, and proposed applications, after a protracted and sometimes difficult decade in development. This was to support a more consensual approach to revalidation policy before its launch in 2012. METHOD: In 2010 and 2011, the authors carried out a critical discourse analysis of interviews with 31 medical and legal revalidation policy makers. These individuals represented the main stakeholder bodies, including the General Medical Council, Academy of Medical Royal Colleges, British Medical Association, National Health Service Employers, and the departments of health from across the United Kingdom. RESULTS: The authors identified two overarching discourses: regulation and professionalism, held together by patients as "discursive glue." Regulation frames revalidation as a way to identify "bad apples," requiring a summative approach and minimum standards. Professionalism looks to revalidation as a process by which all doctors improve, requiring evolving standards and a developmental model. CONCLUSIONS: These two discourses were not mutually exclusive; indeed, most interviewees used them interchangeably. However, they are in some regards at odds. Their coexistence has been supported by a shared discursive formation around patients. Yet the authors found little patient-centered policy in revalidation in its current form. The authors concluded that patients need to be recognized, making them present with an active voice. They also stressed the importance of established and ongoing evaluation of medical regulation as a policy and process.

From anatomical 'competence' to complex capability. The views and experiences of UK tutors on how we should teach anatomy to medical students.

Developments in clinical education have recently challenged the identity of anatomy teaching and learning, leading to high profile debate over the potential implications for the competence levels of new doctors. However, the emphasis remains on methods of teaching, rather than a review of what well-rounded anatomical learning actually entails, and how teaching can address contemporary learning needs. This paper identifies and addresses some of these issues, drawing on expert views captured in qualitative research with anatomy tutors at twenty different medical schools in the UK. Three main themes emerging from our analysis are described: anatomy as a subject matter, the challenges of teaching or learning anatomy, and the use of teaching methods. We also detail how inductive analysis generated new hypotheses worthy of further consideration. These fall into two key categories: (1) improving anatomy curriculum design and (2) advancing anatomy education research.

Out of the dissecting room: news media portrayal of human anatomy teaching and research.

Radical changes in medical research and education have recently led to a number of innovative developments in terms of how human anatomy is represented and understood. New ways of introducing medical students to anatomy (including living anatomies and virtual simulations) have provoked widespread debate, with discussion of their relative merits compared to more traditional approaches that use cadaveric dissection. Outside the field of medicine, in the wider public sphere, the practice of anatomical study may often seem mysterious. The dissemination of news on anatomy, we contend, is central to the question of how medical researchers and educators engage with the public. Our analysis of news media coverage in the UK demonstrates that news-making, by giving prominence to certain facts, themes and images, serves to mask issues about anatomy and its practices that need debate. We examine the ways in which news media, through processes of selection and the 'framing' of issues, may perform an agenda-setting role. We draw attention to the use of positive 'awe and amazement' frames including 'miracles of modern science', 'medical heroes', and 'gifts of life', alongside more negative 'guts and gore' coverage including 'Frankenstein', 'Brave New World' and 'Rape of the Body' frames that concentrate on high profile scandals associated with the use and misuse of human bodies, tissues and parts. We also highlight the selective use of commentaries from members of the medical profession, which are more prevalent in positive 'awe and amazement' stories than in stories with negative coverage. We conclude by arguing for greater collaboration between journalists on the one hand, and medical educators and researchers on the other, in the making of news in order to provide portrayals of anatomy which bear a closer relationship to the everyday reality of professional work.